Methods and apparatus to monitor, verify, and rate the performance of airings of commercials

ABSTRACT

Methods and apparatus to monitor, verify, and rate the performance of airings of commercials are disclosed. An example method includes analyzing received advertisement detection information associated with the advertisement, the advertisement detection information detected from a presentation of the advertisement; identifying a buy order corresponding to the presentation of the advertisement based on the advertisement detection information; determining a purchased ratings value from the buy order; comparing the purchased ratings value of the buy order to received ratings information corresponding to the presentation of the advertisement to determine whether the advertisement was presented as indicated in the buy order; and generating a performance monitoring report using the buy order and the advertisement detection information to indicate whether the ratings information is less than the purchased ratings value.

RELATED APPLICATIONS

This patent arises from a continuation of U.S. patent application Ser.No. 12/350,223 filed Jan. 7, 2009, entitled “METHODS AND APPARATUS TOMONITOR, VERIFY, AND RATE THE PERFORMANCE OF AIRINGS OF COMMERCIALS,”and claims the benefit of U.S. Provisional Patent Application No.61/019,501, filed Jan. 7, 2008, entitled “METHODS AND APPARATUS TOMONITOR, VERIFY, AND RATE THE PERFORMANCE OF AIRINGS OF COMMERCIALS.”U.S. patent application Ser. No. 12/350,223 and U.S. Provisional PatentApplication Ser. No. 61/019,501 are hereby incorporated by reference intheir entireties.

FIELD OF THE DISCLOSURE

The present disclosure relates generally to media monitoring and, moreparticularly, to methods and apparatus to monitor, verify, and rate theperformance of airings of commercials.

BACKGROUND

When advertisers purchase advertising exposure (e.g., a commercialcampaign on a television network), a contract is negotiated thatidentifies details and/or benchmarks for how the advertisement is to bepresented. For example, the contract may specify the time of day duringwhich an advertisement is to be presented, the type of programming(e.g., television program) during which an advertisement is to bepresented, the type of programming with which an advertisement is not tobe presented, how much time is expected to pass between advertisementpresentations, etc. Advertisement handlers (e.g., a television network)must then manage the presentation of advertisements to meet therequirements defined by the contract.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example performance verifier apparatus to generatecommercial performance verification reports based on commercialdetection information, program schedule information, and commercial buyinformation.

FIG. 2 illustrates an example performance monitoring report generatorapparatus to generate performance monitoring reports based oninformation from the example performance verification apparatus of FIG.1, ratings data, and user guideline information.

FIG. 3 is a flow diagram representative of machine readable instructionsthat may be executed to implement the performance monitoring reportgenerator apparatus of FIG. 2.

FIG. 4 is a user login user interface screen to authenticate registeredusers and grant those users access to the example performance monitoringreport generator apparatus of FIG. 2.

FIG. 5 is a search user interface screen to enable users to requestreports from the example performance monitoring report generatorapparatus of FIG. 2.

FIG. 6 is a system administrator user interface screen to enableadministrator-level users to create advertising agency accounts.

FIG. 7 is a user records user interface screen to display registeredusers associated with different advertisement agencies.

FIG. 8 is an advertiser/product records user interface screen to showadvertiser and product information for different advertisement agencies.

FIG. 9 is an advertisement campaign user interface screen to listadvertising campaigns from different advertisers.

FIG. 10 is an entry/edit user interface screen to enable a user to enterestimated information related to commercial airings of an advertisementcampaign.

FIG. 11 is a ratings methodology user interface screen to enable a userto specify a particular ratings methodology to use for rating aparticular advertisement campaign.

FIG. 12 is an implementation guidelines user interface screen to enablea user to specify rules for how commercials are to be broadcast.

FIG. 13 is an add/edit traffic user interface screen to enable a user toadd or edit commercials for different advertisement campaigns.

FIG. 14 is a multi-market view total user interface screen to enable auser to view total advertisement campaign performance information.

FIG. 15A is a first part of a multi-market view daypart user interfacescreen to enable a user to view daypart advertisement campaignperformance information.

FIG. 15B is a second part of a multi-market view daypart user interfacescreen to enable a user to view daypart advertisement campaignperformance information.

FIG. 16A is a first part of a multi-market view weekly user interfacescreen to enable a user to view weekly advertisement campaignperformance information.

FIG. 16B is a second part of a multi-market view weekly user interfacescreen to enable a user to view weekly advertisement campaignperformance information.

FIG. 17 is a single-market performance user interface screen to displayperformance information of a single estimate in a single market.

FIG. 18 is a single-market weekly performance user interface screen todisplay performance information of a single estimate in a single marketby week.

FIG. 19 is a station view total user interface screen to display totalestimated versus actual rating and gross rating point (GRP) data for aparticular station.

FIG. 20 is a station view weekly user interface screen to displayestimated versus actual rating and GRP data for a particular station byweek.

FIG. 21A is a first part of a station view spot detail user interfacescreen to display detailed information about detected and bought spotsfor a particular station.

FIG. 21B is a second part of a station view spot detail user interfacescreen to display detailed information about detected and bought spotsfor a particular station.

FIG. 22 is a flash report user interface to enable a user to select rowsof data from different reports and send the data to a recipient party.

FIG. 23 is a guidelines violation report default user interface screento display an overview of violation information associated with airedcommercials.

FIG. 24A is a first part of a station violation detail user interfacescreen to display detailed violation information associated with airedcommercials.

FIG. 24B is a second part of a station violation detail user interfacescreen to display detailed violation information associated with airedcommercials.

FIG. 25A is a first part of an implementation guidelines user interfacescreen to allow a user to configure the detection of violations of theguidelines specified in a buy.

FIG. 25B is a second part of an implementation guidelines user interfacescreen to allow a user to configure the detection of violations of theguidelines specified in a buy.

FIG. 26 is an example multi-market view user interface screen thatincludes an audit index.

FIG. 27 is a violations report user interface screen that displaysdetailed information about guideline violations.

FIG. 28 is a block diagram of an example processor system that may beused to implement some or all of the example methods and apparatusdescribed herein.

FIG. 29 is an implementation of report user interface for displayinginformation related to maximum exposure for aired commercials.

FIG. 30 is an implementation of report user interface for displayinginformation related to maximum exposure for aired commercials.

FIG. 31 is an implementation of report user interface for displayinginformation related to maximum exposure for aired commercials.

FIG. 32 is an implementation of report user interface for displayinginformation related to maximum exposure for aired commercials.

FIGS. 15A and 15B represent a single FIGURE and are referenced as FIG.15 throughout the specification. FIGS. 16A and 16B represent a singleFIGURE and are referenced as FIG. 16 throughout the specification. FIGS.21A and 21B represent a single FIGURE and are referenced as FIG. 21throughout the specification. FIGS. 24A and 24B represent a singleFIGURE and are referenced as FIG. 16 throughout the specification. FIGS.25A and 25B represent a single FIGURE and are referenced as FIG. 25throughout the specification.

DETAILED DESCRIPTION

Although the following discloses example apparatus and systemsincluding, among other components, software executed on hardware, itshould be noted that such apparatus and systems are merely illustrativeand should not be considered as limiting. For example, it iscontemplated that any or all of these hardware and software componentscould be embodied exclusively in hardware, exclusively in software, orin any combination of hardware and software. Accordingly, while thefollowing describes example methods, apparatus, and systems, personshaving ordinary skill in the art will readily appreciate that theexamples provided are not the only way to implement such methods,apparatus, and systems.

In general, the example methods and apparatus described herein may beused to monitor, verify, and rate the performance of airings ofcommercials. Although the example methods and apparatus are describedherein using television commercials, the example methods and apparatusmay be used in connection with commercials presented via differentmediums including, for example, radio. As described below, the examplemethods and apparatus described herein are configured to generate,organize and present performance information related to the monitoring,verification and rating of aired commercials. The performanceinformation is generated based on user-provided information thatspecifies how advertisement campaigns are expected to perform. Inaddition, the example methods and apparatus described herein enableusers to specify how commercials are to be monitored and analyzed. Theperformance information is organized and presented using different userinterface screens that enable users to specify different criteriaassociated with how the performance information is presented. Theexample implementation described herein enables advertising handlers tomonitor advertising presentation while a campaign is in progress toenable adjustments to be made to the campaign to meet the specificationsof the advertiser.

To verify the airing of commercial advertisements, the example methodsand apparatus described herein receive user-provided estimateinformation describing purchase order or buy specifics for eachcommercial. The purchase order or buy specifics are then used todetermine whether the commercials were aired as ordered and whether theyreached a target demographic audience. In some example implementations,the example methods and apparatus described herein are implemented inconnection with example systems, methods and apparatus described in U.S.provisional patent application No. 60/976,692, filed on Oct. 1, 2007,and titled “Systems and Methods to Associate Related Market BroadcastDetections into National Detections,” U.S. patent application Ser. No.12/242,192, filed on September 2008, and titled “Systems, Apparatus andMethod to Associate Related Market Broadcast Detections with aMulti-Market Media Broadcast,” and U.S. Pat. No. 7,039,931, issued onMay 2, 2006, and titled “Multi-Market Broadcast Tracking, Management andReporting Method and System,” all of which are hereby incorporated byreference herein in their entireties.

In the illustrated examples described herein, commercial advertisementscan be bought on a per spot or per national basis. A spot refers to thebroadcast source for the television (TV) programming being discussed. Aspot commercial is a commercial run on a local broadcaster, such asWABC. A spot rating is the rating for that local broadcaster. A nationalrefers to a commercial run in connection with TV programming from anational TV broadcast network (e.g., ABC, CBS) or national cable network(e.g., Life, USA, Sci-Fi, etc.). The commercial (e.g., spots ornationals) can be bought by advertisers or advertising agencies. Anadvertiser is a product manufacturer or service provider that is thesource of goods or services being advertised. Example advertisers arecar manufacturers, food manufacturers, internet service providers,entertainment service providers, etc. An advertising agency is an agencythat may be hired by an advertiser to create and/or manageadvertisements including the purchasing of advertisement spots ornationals to air the advertiser's advertisements.

When an advertiser or advertising agency purchases a spot or a national,it is referred to as a buy (e.g., a purchase order). A buy is aninstruction from an advertiser (or advertising agency) to a broadcasterto run a commercial or set of commercials over a set time or series oftimes with set rules (i.e., guidelines for airing commercials). Buyinformation (or purchase order information) includes estimateinformation to estimate airing specifics of a particular advertisementincluding, for example, goals for reach, frequency, demographics,average rating and market, among other metrics. The example methods andapparatus described herein are configured to generate reports indicatingwhether commercial airing guidelines (as set forth by rules of the buyinformation) have been met or violated for corresponding commercialairings.

The terms estimate and campaign are used interchangeably to refer to asubcomponent (e.g., reach, frequency, demographics, average rating,market, etc.) of the estimate of a media plan. A buy is theimplementation instruction of an estimate (or campaign). The termtraffic refers to instructions specifying what commercials may be airedwhen fulfilling a buy for a campaign. When a commercial is permitted tobe run as part of a campaign, it is said to be trafficked to thatcampaign. When monitoring airings of commercials, a match occurs foreach detection of an encoded commercial that has been matched to a buy(and, thus, to a campaign). A detected commercial that does not matchany buys (and, thus, does not match any campaigns) is referred to asunordered. An allowance refers to a window of time around the buyinstructions during which a commercial may be aired and still be matchedto a buy.

Turning to FIG. 1, an illustrated example performance verifier apparatus100 may be used to generate commercial performance verification reports102 based on commercial detection information 104 (collected programname code, electronic signature, or diary information), program scheduleinformation 106, and commercial buy information 110. The exampleperformance verifier apparatus 100 is an analytic system that receivesthe commercial detection information 104, the program scheduleinformation 106, and the commercial buy information 110 to generate theperformance verification reports 102. The commercial performanceverification reports 102 describe how a user (e.g., an advertisingagency or an advertiser) expected a commercial to perform and the actualperformance that was detected based on the airing(s) of that commercial.

FIG. 2 illustrates an example performance monitoring report generatorapparatus 200 to generate performance monitoring reports 202 based oninformation from the example performance verification apparatus 100 ofFIG. 1, ratings data 204, and user guideline information 206. Theperformance monitoring reports 202 are used to report whethercommercials are running when ordered, whether the commercials are runduring time periods getting sufficiently high ratings, and whether theairings of the commercials are within constraints or limits specified byan advertiser or advertising agency. Each of the performance monitoringreports 202 can include one or more of ratings report information 208,verification report information 210, and guidelines report information212. While the performance verifier apparatus 100 of FIG. 1 verifieswhether commercials ran when ordered based on buys (e.g., a particularcommercial ran in 5 spots during the week of March 15), the performancemonitoring report generator apparatus 200 informs users whether theirbuy contracts are being correctly fulfilled by verifying whethercommercials ran when ordered based on buys (e.g., the buy information110), based on whether the commercials were received by the expectedaudience (e.g., a demographic group of males between the ages of 18-54,and a 3.6 average rating), and based on whether the commercials were runas instructed (e.g., five spots per week during the week of March 15,but never during the Jerry Springer show).

In the illustrated examples described herein, performance monitoringreport generator apparatus 200 presents performance-related data using aseries of user interface screens described below that are arranged in adrill-down hierarchy ranging from high level summary data to specificinstance datum. This drill-down hierarchy organization allows users torelatively quickly identify under-performing buys and relatively quicklyisolate the cause of the underperformance. The performance monitoringreport generator apparatus 200 implements a color-coded index (e.g., ared index) to highlight data indicative of under-performance, whichallows users to identify performance thresholds against which differentdata (e.g., rows of a data grid) is compared.

In the illustrated examples described herein, the performance monitoringreport generator apparatus 200 primarily presents performance-relatedinformation using data grids. At each level of the drill-down hierarchy,the performance monitoring report generator apparatus 200 presentsdifferent data grids based on different combinations of variables/data.The processes implemented using the performance monitoring reportgenerator apparatus 200 support the various ways that users think aboutdata off-line or think about the concept of measuring or quantifyingperformance.

To enhance communications and foster collaborative work environments,the performance monitoring report generator apparatus 200 enables usersto select particular data from reports (e.g., using the flash reportuser interface described in connection with FIG. 22) and send theselected data to others via, for example, email. Alternatively oradditionally, users may download reports into spreadsheet or PDFformats, which may be redistributed to others.

In the example implementations described herein, the performancemonitoring report generator apparatus 200 uses the commercial detectioninformation 104 (FIGS. 1 and 2) to associate the detection ofcommercials with an audience measurement and a specific buy and toreport the associated information. The performance monitoring reportgenerator apparatus 200 enables users to submit a particular audiencedemographic and a ratings value expected to be achieved for a timeperiod associated with a particular commercial along with buyinformation. For example, for each line of a spot-based and/ornational-based buy (e.g., each ordered airing of a commercial), theremay be one or more demographic/rating pairs. In the exampleimplementations described herein, up to four demographic/rating pairsmay be used. The four demographic types are referred to as the primary,secondary, tertiary, and quaternary demographics. In the exampleimplementations described herein, for a spot-based buy, a user mustprovide a household ratings estimate as well. In the illustratedexamples described herein, the performance monitoring report generatorapparatus 200 matches each commercial detection to a single buy. Thus,the performance monitoring report generator apparatus 200 does notgenerally associate a commercial detection with a product, but insteadwith a buy so that the performance of the commercial detection can bedetermined and quantified based on the buy information.

FIG. 3 is a flow diagram representative of machine readable instructionsthat may be executed to implement the performance monitoring reportgenerator apparatus 200 of FIG. 2. Some or all of the blocks of each theflow diagram may be representative of machine readable instructions thatmay comprise one or more programs for execution by one or moreprocessors (e.g., the processor 2812 of FIG. 28), one or morecontrollers, and/or any other suitable devices. The one or more programsmay be embodied in software stored on a tangible medium such as, forexample, one or both of the memories 2824 and 2825 of FIG. 28. Theentire program or programs and/or portions thereof could alternativelybe executed by a device other than the processor 2812 and/or may beembodied in firmware or dedicated hardware in any desired manner (e.g.,implemented using an application specific integrated circuit (ASIC), aprogrammable logic device (PLD), a field programmable logic device(FPLD), discrete logic, etc.). Also, some or all of the operations ofthe flow diagram of FIG. 3 may be implemented manually. Further,although the example method is described with reference to the flowdiagram illustrated in FIG. 3, many other techniques for implementingthe example methods and apparatus described herein may alternatively beused. For example, with reference to the flow diagram illustrated inFIG. 3, the order of execution of the blocks may be changed, and/or someof the blocks described may be changed, eliminated, combined and/orsubdivided into multiple blocks.

In the illustrated example of FIG. 3, the performance monitoring reportgenerator apparatus 200 receives a detection record 301 (block 302) fromthe performance verifier apparatus 100 of FIG. 1. The detection record301 is generated by the performance verifier apparatus 100 and includesinformation indicating that buys were matched with commercial detectionsto verify whether or not the buys were fulfilled. The information in thedetection record is organized using the following headers: 1) time, 2)date, 3) station, 4) market, 5) buy (if matched to a buy), and 6)allowance status (before, during, or after an ordered time or timerange).

If the example implementation includes both local and nationalcommercial monitoring, the system may determine which type of commercialis associated with the detection record at this time. Using thedetection record, the performance monitoring report generator apparatus200 determines buy information (block 304). For example, if thedetection record 301 indicates that a corresponding detected commercialis unmatched to a specific buy (e.g., it may have been generally matchedto a product by the performance verification apparatus 100 of FIG. 1),the performance monitoring report generator apparatus 200 associates theunordered detection to a single buy. To match the commercial detectionof the detection record 301 to a specific buy, the performancemonitoring report generator apparatus 200 1) identifies all buys towhich the detected commercial is trafficked; 2) identifies which buysinclude the detection record's market; 3) if only one buy includes thedetection record's market, then assign the detection record 301 to thatbuy; 4) if more than one buy matches the market of the detection record,then identify which buys include the market and station for thedetection record 301; 5) if only one buy includes the market and stationof the detection record, then assign the detection record 301 to thatbuy; 6) if more than one buy matches the detection record's market andstation, then identify which buys include the market, station andincluded dates w/in two weeks of for the detection; 7) if only one buyincludes the detection record's market, station, and included dates,then assign the detection record 301 to that buy; 8) if more than onebuy matches the detection record's market, station, and included dates,then identify which buys include the market, station, and week-of forthe detection record 301; 9) if only one buy matches the detectionrecord's market, station, and week-of, then assign the detection record301 to that buy; 10) if more than one buy matches the detection record'smarket, station, and week-of, determine which buy has the earliest startdate; 11) if only one buy has the earliest start date, then assign thedetection record 301 to that buy; 12) if more than one buy share thesame earliest start date, then determine which buy has the lowestestimate number; and 13) assign the detection record 301 to the buy withthe lowest estimate number.

The estimate or campaign for the buy determined at block 304 is then set(block 306) based on the buy information and any other user-submittedinformation. The performance monitoring report generator apparatus 200then converts the detection time of the detection record into quarterhour segments (block 308). For example, a detection time of (09:12:56)is converted to a quarter hour (09:00-09:14:59). Users are allowed toconfigure a permitted allowance for matching detections to buys. Forexample, with a 2 minute allowance, the system will match a 4:58:36detection to a 5:00-7:00 buy. When an allowance match is made, thedetection quarter hour is set as the nearest quarter hour within the buyinstructions. Each matched detection record reports a character (e.g.,B, D or A) to indicate if the detection was [B]efore the buy, but in theallowance, [D]uring the buy, or [A]fter the buy, but in the allowance.The character is used to determine the closest quarter hour within thebuy.

The performance monitoring report generator apparatus 200 then selects aratings source (block 310). For example, the performance monitoringreport generator apparatus 200 of the illustrated example loads NielsenAudience Measurement data into a database as those measurements are madeavailable to the public. There are different sources for differentmarkets and market coverage. To select a source, the performancemonitoring report generator apparatus 200 of the illustrated exampleevaluates how a user configured the buy and the ratings sourcesavailable to determine which is the best ratings source to use. Forexample, if the Viewers In Profile (VIP) report from Nielsen MediaResearch is available for a detection market and date, it is used. If itis not available, the Nielsen Media Research Live+7 report is used. Ifthe Live+7 report is not available, the Nielsen Media Research Overnightreport is used. If none of the reports are available, a ratings sourcemay not be set and modeling rules may be used (as described in furtherdetail below). Additionally, a user may specify that modeling rulesshould be used even when a report is available.

The performance monitoring report generator apparatus 200 then matchesthe detection to measurements (block 312). Using the quarter hour anddate of the detection and the rating source previously selected, thesystem retrieves the appropriate impressions and universal estimate. Atelevision rating is calculated using ratings formulas (e.g., ratingsformulas published by Nielsen Media Research). If no rating source wasselected, this block may be skipped.

After matching detections to measurements (block 312), the ratingsinformation is validated (block 314). For example, the ratingsinformation must have sufficient in-tab and must be statisticallysignificant. Then, the performance monitoring report generator apparatus200 of the illustrated example applies special buy rules and changesratings values as specified by the rules (block 316). For example, if abuy includes a “-s” value in a line of the buy, the system is instructedto use the target ratings value in place of the actual ratings value inits reports (Dash S feature). In another example, if a buy is for anexact time either on the hour or half hour, the system is instructed toaverage the two adjacent quarter hours and use that average in place ofthe actual ratings value (Break-buy feature).

Then, the performance monitoring report generator apparatus 200 appliesmodeling rules (block 318). In the illustrated example, modeling is theestimation of a television rating when no true rating is available. Forexample, in a first market, no overnight demographic ratings areavailable and in another market no VIP ratings are released. When noratings information is available, model ratings are used. For example,model demographics may be estimated based on households and/or allvalues of information may be estimated. For example, when demographicratings are not available for a market, a modeling demographic iscalculable by taking the ratio between the estimated household ratingprovided in the buy and the actual household rating and, then, applyingit to the estimated demographic rating to achieve a modeled demographicrating. In another example, when no ratings source is available forseveral months, targeted ratings values from the buy may be used as theactual values.

Then, the performance monitoring report generator apparatus 200 storesthe detection record and corresponding ratings information to a database(block 320). The detection record and/or ratings information can beretrieved from the database to analyze the results of a campaign.

FIGS. 4-24 are graphical user interfaces that allow users to access anduse the example performance monitoring report generator apparatus ofFIG. 2. The example graphical user interfaces allow users to monitorwhether the television commercial campaigns they have bought are airedas ordered and get the market weight and pressure desired byadvertisers.

FIG. 4 is a user login user interface screen to authenticate registeredusers and grant those users access to the example performance monitoringreport generator apparatus of FIG. 2. Access to the registered userhomepage is restricted and is controlled by the log-on function from thedefault homepage. The screen contains four basic modules: marketingmessages 402, system messages 404, a search function 406, andadministration links 408. In the illustrated example, the searchfunction 406 and the administrative links 408 are based on the user'suser-type. In the illustrated example, an agency administration link 410links to a system administrator user interface (e.g., the systemadministrator user interface screen of FIG. 6). A campaignadministration link 412 links to an advertisement campaign userinterface (e.g., the advertisement campaign user interface screen ofFIG. 9). A traffic administration link 414 links to a traffic userinterface screen (e.g., the traffic user interface screen of FIG. 13).

FIG. 5 is a search user interface screen to enable users to requestreports from the example performance monitoring report generatorapparatus of FIG. 2. The search function is a form in which the userselects information from pull-downs, pop-up calendars, and radiobuttons. The data grid of the search results is based on the searchcriteria entered and the user's permissions.

FIG. 6 is a system administrator user interface screen to enableadministrator-level users to create advertising agency accounts. Withinthe system administrator user interface screen the user creates agencyaccounts, adds advertiser and product records to those accounts, andthen creates the initial user accounts for the agency. The systemadministrator user interface screen of the illustrated example includestabs that allow a user to select which type of record to input. Theprint function causes a comma separated values (CSV) file to bedownloaded containing the current data grid information.

FIG. 7 is a user records user interface screen to display registeredusers associated with different advertisement agencies. The example userrecords user interface screen of FIG. 7 is displayed when a user selectsan advertiser/products tab of the system administrator user interfacescreen of FIG. 6. The print function causes a CSV file to be downloadedcontaining the current data grid information.

FIG. 8 is an advertiser/product records user interface screen to showadvertiser and product information for different advertisement agencies.The example user records user interface screen of FIG. 8 is displayedwhen a user selects a users tab of the system administrator userinterface screen of FIG. 6. The agency drop down menu allows users toselect which agency's users are displayed. As items are selected fromthe drop down menu, the data grid changes. The print function causes aCSV file to be downloaded containing the current data grid information.

FIG. 9 is an advertisement campaign user interface screen to listadvertising campaigns from different advertisers. When a user selects acampaign and chooses an enter new campaign button, the user is taken toa campaign/estimate screen (e.g., the campaign/estimate screen of FIG.10) where the user can input new campaign information. In theillustrated example, when a user clicks on a number in an estimatecolumn, the user is taken to a campaign/estimate screen that ispre-populated with information from the selected campaign so that theuser can edit the information.

FIG. 10 is an entry/edit user interface screen to enable a user to enterestimated information related to commercial airings of an advertisementcampaign. When a user is creating a new campaign, the entry/edit userinterface screen is blank when a user reaches the screen. When a user isediting a campaign, the entry/edit user interface screen ispre-populated with information about the campaign to be edited. Theentry/edit user interface screen includes several tabs. A list tabreturns the user to the campaign list screen. A ratings methodology tabtakes the user to a ratings methodology screen (e.g., the ratingsmethodology user interface screen of FIG. 11) where the user selects theratings methodology for the display campaign. An implementationguidelines tab takes the user to an implementations guidelines screen(e.g., the implementation guidelines user interface screen of FIG. 12)where the user sets the implementation guidelines for the displayedcampaign.

FIG. 11 is a ratings methodology user interface screen to enable a userto specify a particular ratings methodology to use for rating aparticular advertisement campaign. The ratings methodology userinterface screen includes a user election radio button for each type ofmetered market that allows a user to select whether modeled data shouldbe used. The example ratings methodology user interface screen alsoincludes a radio button to allow a user to select whether to model datain unmetered markets (e.g., where only VIP information is available). Auser field and an approval date field enable a user to inputidentification information about who changed/set the values and the dateon which the change/setting was made. A save button allows a user tosave the data to the database.

FIG. 12 is an implementation guidelines user interface screen to enablea user to specify rules for how commercials are to be broadcast. A copyguidelines from another campaign drop down menu allows a user to selecta previously existing campaign to populate the fields of the form.

FIG. 13 is an add/edit traffic user interface screen to enable a user toadd or edit commercials for different advertisement campaigns. A user(e.g., an administrator) assigns commercials to campaigns. Commercialsare referenced by an industry standard commercial identifier (ISCI). Forexample, each commercial has a unique ISCI. In the illustrated example,traffic is entered at the campaign level. The approved ISCI fieldcontains ISCIs for traffic that exists for the campaign and allows auser to enter approved ISCIs for the campaign.

FIG. 14 is a multi-market view total user interface screen to enable auser to view total advertisement campaign performance information. Usersarrive at the multi-market view total user interface screen when asearch is executed on the authenticated user homepage. The data grid ofthe multi-market view are based on the search criteria. Each row of thedata grid of the illustrated example is for a unique combination ofadvertiser, product, market, and estimate. A user can execute a newsearch by changing the form values and then selecting a fetch databutton. A user can select one of a primary or secondary demographicassociated with a campaign by selecting a radio button next to a desireddemographic and selecting the fetch data button.

In the illustrated example, each campaign has an index that iscalculated by dividing achieved gross rating points (GRPs) into the buyGRPs. A less-than display instructs the system to display only thosecampaigns that are performing below the numeric threshold set by theuser. A less-than red index instructs the system to highlight in red thecells of the campaigns performing below the numeric threshold set by theuser. For example, in the illustrated example, the index for HewlettPackard is highlighted in red.

FIG. 15 is a multi-market view daypart user interface screen to enable auser to view daypart advertisement campaign performance information. Themulti-market view daypart user interface screen displays how a singleestimate has performed across many markets by daypart. In theillustrated example, each campaign has a primary and a secondarydemographic assigned to it during set-up. The selection of a demographicaffects the information presented in the example multi-market viewdaypart user interface screen. In the illustrated example, the Totalvalue for each record is a hyperlink to a multi-market view total screen(e.g., the multi-market view total screen of FIG. 17). In theillustrated example, the Weekly value for each record is a hyperlink toa multi-market view weekly screen (e.g., the multi-market view weeklyscreen of FIG. 16).

FIG. 16 is a multi-market view weekly user interface screen to enable auser to view weekly advertisement campaign performance information. Themulti-market view weekly user interface screen of the illustratedexample displays how an estimate has performed across many markets byweek.

FIG. 17 is a single-market performance user interface screen to displayperformance information of a single estimate in a single market. Thesingle-market performance user interface screen shows the user theperformance of a single estimate in a single market. An exampleimplementation audit hyperlink causes the user to navigate to aguidelines violation report screen (e.g., the guidelines violationreport screen of FIG. 23). An example station call letter hyperlinkcauses the user to navigate to a station view total screen (e.g., thestation view total screen of FIG. 19) where the data is narrowed to aselected station. An example GRP index hyperlink causes the user tonavigate to the station view total screen.

FIG. 18 is a single-market weekly performance user interface screen todisplay performance information of a single estimate in a single marketby week. A day of the week data grid is populated based on a day of theweek delivery drop down menu. For example, the drop down menu has adefault value of cumulative, which indicates that an aggregation ofvalues for all weeks in the campaign (limited by the search request)should be shown. The example drop down menu includes values for thecompleted weeks for the campaign. Selecting a weekly value from the dropdown menu causes the day of the week data grid to display informationfor the selected week.

FIG. 19 is a station view total user interface screen to display totalestimated versus actual rating and GRP data for a particular station. Inthe illustrated example, the station view total user interface screen ispopulated based on the last search conducted by a user.

FIG. 20 is a station view weekly user interface screen to displayestimated versus actual rating and GRP data for a particular station byweek. In the illustrated example, the station view weekly user interfacescreen is populated based on the last search conducted by a user.

FIG. 21 is a station view spot detail user interface screen to displaydetailed information about detected and bought spots for a particularstation. In the illustrated example, the station view spot detail userinterface screen displays a line-by-line detail of detected spots andbought spots including details of the buy, details of the airing of thespot, and ratings information about the airing compared with theestimated ratings information. The example station view spot detail userinterface screen includes a missed units only function that allows auser to request that only information for spots that were bought but notdetected be displayed. The station view spot detail user interfacescreen of the illustrated example includes a flash report function thatallows the user to select rows from the data grid and send informationfrom those rows to others via email or any other communication medium. Auser can select one or more rows to be sent in a flash report. When arow is selected, the row is highlighted in yellow. When a row isselected a second time, the row is de-selected. After selecting a row,the user selects a flash report button, which causes a flash report userinterface to be displayed (e.g., the flash report user interface screenof FIG. 22).

FIG. 22 is a flash report user interface screen to enable a user toselect rows of data from different reports and send the data to arecipient party. For example, the flash report user interface enables auser to select rows of data from a multi-row data grid of one or more ofthe user interfaces described herein and send those rows of data alongwith comments to others via email. The flash report user interface ofthe illustrated example verifies that the email address input by theuser is properly formed (e.g., includes an account, the @ symbol, and adomain name).

FIG. 23 is a guidelines violation report default user interface screento display an overview of violation information associated with airedcommercials. Users can reach the guidelines violation report defaultuser interface screen by selecting a value displayed in animplementation audit column of a multi-market view total user interface(e.g., the multi-market view total user interface screen of FIG. 14) orby selecting an implementation audit button on a user interface (e.g.,the implementation audit button on the station view weekly userinterface screen of FIG. 20). In the illustrated example, when a userselects a station violation detail hyperlink the user is taken to astation violation detail screen (e.g., the station violation detailscreen of FIG. 24). The guidelines violation report default userinterface screen may include a link to return the user to the userinterface screen from which the user linked to the guidelines violationreport default user interface screen.

FIG. 24 is a station violation detail user interface screen to displaydetailed violation information associated with aired commercials. Thestation violation detail user interface screen may include a link toreturn the user to the user interface screen from which the user linkedto the guidelines violation report default user interface screen.

FIG. 25 is an implementation guidelines user interface screen to allow auser to configure the detection of violations of the guidelinesspecified in a buy. Guideline verification allows a user (e.g., a buyer)to discern mid-buy whether a commercial occurrence should be consideredpart of the buy's fulfillment. Implementation guidelines of theillustrated example are rules concerning how commercials are to beaired. In the illustrated example, broadcasters are contractuallyobligated to follow guidelines specified in the buy. The guidelinesinput by the user in the implementation guidelines user interface screenare compared to the results of detections to determine if the airing ofcommercials complies with the guidelines.

For example, a guideline violation may occur when the wrong creative isdetected for a commercial. In the illustrated example, the rating pointsassociated with the wrong creative are tracked. The number of pointsrequired to meet the requirements of the buy are tracked.

Additionally, an example guideline violation occurs when a commercial isshown during excluded programming. For example, an advertiser mayrequest that a commercial not be shown during a program that does notmatch the advertiser's desired demographic. In the illustrated example,users input the titles for programs that should be excluded. If theprogram during which the commercial was aired is on an excluded programslist, a guideline violation is reported.

Additionally, an example guideline violation occurs when a commercial isshown on the wrong day of the week.

Additionally, an example guideline violation occurs when commercials arenot separated by a desired amount of time. In the illustrated example, adetection date and time for a commercial is compared with otherdetections for the same campaign to determine if a sufficient amount oftime has passed. According to an example implementation, where asufficient amount of time has not passed, ratings for the secondoccurrence are used for index calculation.

Additionally, an example guideline violation occurs when the maximumnumber of airings during a program has been reached for a commercial.For example, once a detection is matched to a campaign, the date, time,and program name are compared to previous detections to determine if amaximum number of spots designated in the buy has been reached.

FIG. 26 is an example multi-market view user interface screen thatincludes an audit index. The audit index identifies the percentage ofreported GRPs that are attributed to commercials that violatedimplementation guidelines (e.g., the implementation guides specified inthe implementation guidelines user interface screen of FIG. 25). Theaudit index value of the illustrated example is a hyperlink to a detailviolations report (e.g., the violations report user interface screen ofFIG. 27).

FIG. 27 is a violations report user interface screen that displaysdetailed information about guideline violations. In the illustratedexample, every violation for a campaign is displayed under an applicableheading. According to the illustrated example, the complete detectionrecord is displayed and violation elements are highlighted in red andratings for the violating detection are highlighted in yellow.

FIG. 28 is a block diagram of an example processor system 2810 that maybe used to implement the apparatus and methods described herein. Asshown in FIG. 28, the processor system 2810 includes a processor 2812that is coupled to an interconnection bus 2814. The processor 2812 maybe any suitable processor, processing unit or microprocessor. Althoughnot shown in FIG. 28, the system 2810 may be a multi-processor systemand, thus, may include one or more additional processors that areidentical or similar to the processor 2812 and that are communicativelycoupled to the interconnection bus 2814.

The processor 2812 of FIG. 28 is coupled to a chipset 2818, whichincludes a memory controller 2820 and an input/output (I/O) controller2822. As is well known, a chipset typically provides I/O and memorymanagement functions as well as a plurality of general purpose and/orspecial purpose registers, timers, etc. that are accessible or used byone or more processors coupled to the chipset 2818. The memorycontroller 2820 performs functions that enable the processor 2812 (orprocessors if there are multiple processors) to access a system memory2824 and a mass storage memory 2825.

The system memory 2824 may include any desired type of volatile and/ornon-volatile memory such as, for example, static random access memory(SRAM), dynamic random access memory (DRAM), flash memory, read-onlymemory (ROM), etc. The mass storage memory 2825 may include any desiredtype of mass storage device including hard disk drives, optical drives,tape storage devices, etc.

The I/O controller 2822 performs functions that enable the processor2812 to communicate with peripheral input/output (I/O) devices 2826 and2828 and a network interface 2830 via an I/O bus 2832. The I/O devices2826 and 2828 may be any desired type of I/O device such as, forexample, a keyboard, a video display or monitor, a mouse, etc. Thenetwork interface 2830 may be, for example, an Ethernet device, anasynchronous transfer mode (ATM) device, an 802.11 device, a DSL modem,a cable modem, a cellular modem, etc. that enables the processor system2810 to communicate with another processor system.

While the memory controller 2820 and the I/O controller 2822 aredepicted in FIG. 28 as separate functional blocks within the chipset2818, the functions performed by these blocks may be integrated within asingle semiconductor circuit or may be implemented using two or moreseparate integrated circuits.

FIGS. 25-28 illustrate example implementations of report user interfacesfor displaying information related to maximum exposure for airedcommercials. According to an example implementation, advertisers canrequest reports when a particular demographic exceeds a maximumthreshold for percentage of viewing audience. For example, advertisersof pharmaceuticals, adult beverages, tobacco products, need to ensurethat they are not advertising products to minors. Accordingly, anadvertiser can request a watch to be placed on a particular demographicgroup (e.g., children ages 2-17). Companies can use reports to showstatutory compliance and voluntary self-regulation. Other companies canuse these reports to ensure they are delivering not only to their targetaudience, but also are not delivering a message to an undesired demo.For example, a company may have a female targeted creative it does notwant to air to large male audiences, fearing the female targetedcommercial may diminish the product brand to male consumers. An exampleprocess that may be implemented by software, hardware, and/or anycombination of software and hardware, may include:

1. The User enables a campaign to include a DemoWatch report.

2. The user sets the watch demo.

3. The system collects detections of commercials for the campaign.

4. The system associates the detections with viewer impressions for bothtotal viewers and the watch demo.

5. The system divides watch demo impressions by total viewer impressionsto determine the percentage of watch demo that view the commercialoccurrence.

6. If the demo watch viewership exceeds the threshold, the systemrecords the occurrence as a demo watch violation.

7. The system reports all DemoWatch violations in online and digitaldownload formats.

Although certain methods, apparatus, systems, and articles ofmanufacture have been described herein, the scope of coverage of thispatent is not limited thereto. To the contrary, this patent covers allmethods, apparatus, systems, and articles of manufacture fairly fallingwithin the scope of the appended claims either literally or under thedoctrine of equivalents.

What is claimed is:
 1. A method for monitoring an advertisement, themethod comprising: analyzing received advertisement detectioninformation associated with the advertisement, the advertisementdetection information detected from a presentation of the advertisement;identifying a buy order corresponding to the presentation of theadvertisement based on the advertisement detection information;determining a purchased ratings value from the buy order; comparing, viaa processor, the purchased ratings value of the buy order to receivedratings information corresponding to the presentation of theadvertisement to determine whether the advertisement was presented asindicated in the buy order; and generating a performance monitoringreport using the buy order and the advertisement detection informationto indicate whether the ratings information is less than the purchasedratings value.
 2. A method as defined in claim 1, further comprising:selecting a first ratings source from a plurality of ratings sources,the first ratings source to provide the received ratings information. 3.A method as defined in claim 1, further comprising: attempting toretrieve the ratings information from a first ratings source;determining that the first ratings source is not available; andselecting a second ratings source from the plurality of ratings sources,the second ratings source to provide the received ratings informationinstead of the first ratings source.
 4. A method as defined in claim 1,further comprising: extracting actual presentation information from theadvertisement detection information; matching the actual presentationinformation to expected presentation information in a database of buyorders; and identifying the buy order when the expected presentationinformation associated with the buy order corresponds to the actualpresentation information.
 5. A method as defined in claim 4, wherein theactual presentation information and the expected presentationinformation comprise at least one of a time of the presentation of theadvertisement, a date of the presentation of the advertisement, astation of the presentation of the advertisement, or a market of thepresentation of the advertisement.
 6. An apparatus comprising: aperformance verifier to: analyze received advertisement detectioninformation associated with the advertisement, the advertisementdetection information detected from a presentation of the advertisement,identify a buy order corresponding to the presentation of theadvertisement based on the advertisement detection information,determine a purchased ratings value from the buy order, and compare thepurchased ratings value of the buy order to received ratings informationcorresponding to the presentation of the advertisement to determinewhether the advertisement was presented as indicated in the buy order;and a report generator to generate a performance monitoring report usingthe buy order and the advertisement detection information to indicatewhether the ratings information is less than the purchased ratingsvalue.
 7. An apparatus as defined in claim 6, wherein the performanceverifier is further to select a first ratings source from a plurality ofratings sources, the first ratings source to provide the receivedratings information.
 8. An apparatus as defined in claim 6, wherein theperformance verifier is further to: attempt to retrieve the ratingsinformation from a first ratings source; determine that the firstratings source is not available; and select a second ratings source fromthe plurality of ratings sources, the second ratings source to providethe received ratings information instead of the first ratings source. 9.An apparatus as defined in claim 6, wherein the performance verifier isfurther to: extract actual presentation information from theadvertisement detection information; match the actual presentationinformation to expected presentation information in a database of buyorders; and identify the buy order when the expected presentationinformation associated with the buy order corresponds to the actualpresentation information.
 10. An apparatus as defined in claim 9,wherein the actual presentation information and the expectedpresentation information comprise at least one of a time of thepresentation of the advertisement, a date of the presentation of theadvertisement, a station of the presentation of the advertisement, or amarket of the presentation of the advertisement.
 11. A tangible machinereadable storage device or storage disc comprising instructions that,when executed, cause a machine to at least: analyze receivedadvertisement detection information associated with the advertisement,the advertisement detection information detected from a presentation ofthe advertisement; identify a buy order corresponding to thepresentation of the advertisement based on the advertisement detectioninformation; determine a purchased ratings value from the buy order;compare the purchased ratings value of the buy order to received ratingsinformation corresponding to the presentation of the advertisement todetermine whether the advertisement was presented as indicated in thebuy order; and generate a performance monitoring report using the buyorder and the advertisement detection information to indicate whetherthe ratings information is less than the purchased ratings value.
 12. Astorage device or storage disc as defined in claim 11, wherein theinstructions, when executed, further cause the machine to select a firstratings source from a plurality of ratings sources, the first ratingssource to provide the received ratings information.
 13. A storage deviceor storage disc as defined in claim 11, wherein the instructions, whenexecuted, further cause the machine to: attempt to retrieve the ratingsinformation from a first ratings source; determine that the firstratings source is not available; and select a second ratings source fromthe plurality of ratings sources, the second ratings source to providethe received ratings information instead of the first ratings source.14. A storage device or storage disc as defined in claim 11, wherein theinstructions, when executed, further cause the machine to: extractactual presentation information from the advertisement detectioninformation; match the actual presentation information to expectedpresentation information in a database of buy orders; and identify thebuy order when the expected presentation information associated with thebuy order corresponds to the actual presentation information.
 15. Astorage device or storage disc as defined in claim 14, wherein theactual presentation information and the expected presentationinformation comprise at least one of a time of the presentation of theadvertisement, a date of the presentation of the advertisement, astation of the presentation of the advertisement, or a market of thepresentation of the advertisement.